A color hand gesture database for evaluating and improving algorithms on hand gesture and posture recognition

With the increase of research activities in vision-based hand posture and gesture recognition, new methods and algorithms are being developed. Although less attention is being paid to developing a standard platform for this purpose. Developing a database of hand gesture images is a necessary first step for standardizing the research on hand gesture recognition. For this purpose, we have developed an image database of hand posture and gesture images. The database contains hand images in different lighting conditions and collected using a digital camera. Details of the automatic segmentation and clipping of the hands are also discussed in this paper

Real-time vision-based hand and face tracking and recognition of gesture : a PhD dissertation submitted in partial fulfillment of the requirement for the degree of Doctor of Philosophy (Ph.D.) in Computer Science

In this dissertation, we present the research pathway to the design and implementation of a real-time vision-based gesture recognition system. This system was built based on three components, representing three layers of abstraction: i) detection of skin and localization of hand and face, ii) tracking multiple skin blobs in video sequences and finally iii) recognition of gesture movement trajectories. The adaptive skin detection, the first component, was implemented based on our novel adaptive skin detection algorithm for video sequences. This algorithm has two main sub-components: i) the static skin detector, which is a skin detection method based on the hue factor of the skin color, and ii) the adaptive skin detector which retrains itself based on new data gathered from movement of the user. The results of our experiments show that the algorithm improves the quality of skin detection within the video sequences. For tracking, a new approach for boundary detection in blob tracking based on the Mean-shift algorithm was proposed. Our approach is based on continuous sampling of the boundaries of the kernel and changing the size of the kernel using our novel Fuzzy-based algorithm. We compared our approach to the kernel density-based approach, which is known as the CAM-Shift algorithm, in a set of different noise levels and conditions. The results show that the proposed approach is superior in stability against white noise, and also provides correct boundary detection for arbitrary hand postures, which is not achievable by the CAM-Shift algorithm. Finally we presented a novel approach for gesture recognition. This approach includes two main parts: i) gesture modeling, and ii) gesture recognition. The gesture modeling technique is based on sampling the gradient of the gesture movement trajectory and presenting the gesture trajectory as a sequence of numbers. This technique has some important features for gesture recognition including robustness against slight rotation, a small number of required samples, invariance to the start position and device independence. For gesture recognition, we used a multi-layer feed-forward neural-network. The results of our experiments show that this approach provides 98.71% accuracy for gesture recognition, and provides a higher accuracy rate than other methods introduced in the literature. These components form the required framework for vision-based real-time gesture recognition and hand and face tracking. The components, individually or as a framework, can be applied in scientific and commercial extensions of either vision-based or hybrid gesture recognition systems

Real-time hand tracking using a set of cooperative classifiers based on haar-like features

In this paper we discuss the importance of the choice of features in digital image object recognition. The features can be classified as invariants or noninvariants. Invariant features are robust against one or more modifications such as rotations, translations, scaling and different light (illumination) conditions. Non-invariant features are usually very sensitive to any of these modifiers. On the other hand, non-invariant features can be used even in the event of translation, scaling and rotation, but the feature choice is in some cases more important than the training method. If the feature space is adequate then the training process can be straightforward and good classifiers can be obtained. In the last few years good algorithms have been developed relying on noninvariant features. In this article, we show how non-invariant features can cope with changes even though this requires additional computation at the detection phase. We also show preliminary results for a hand detector based on a set of cooperative Haar-like feature detectors. The results show the good potential of the method as well as the challenges to achieve real-time detection.

Effective ways of encouraging teachers to design and use ITS: Feature analysis of intelligent tutoring systems authoring tools

Intelligent tutoring systems (ITSs) have been proven effective in supporting students' learning activities, but the actual utilization of ITSs has not been confirmed. Delegation of development tasks from developers to teachers through the use of ITS authoring tools (ITSATs) is not promoting rapid progress in this area of research. Designing ITSs using ITSATs by teachers seems to be difficult to realize. This could be affected by insufficient features in ITASTs which meet teachers' requirements. This paper presents feature analysis of ITSATs. In this study, we examined two categories of features: authoring environment and lesson content creation. The study focuses on three ITSAT namely CTAT, ASPIRE and Assistment Buider. We investigated the availability of such features in those ITSAT and propose ITSAT features which could meet teachers' requirements in using ITSAT to design ITS

Demo: An automated face enrolment and recognition system across multiple cameras on CCTV networks

In this paper, we present an architecture for a video analytics framework, specifically designed for automatic enrollment and subsequent re-identification of faces on a network of cameras. The proposed system can be used as an assistive tool for applications such as passenger screening at airports as passengers walk through various sections of the airport

Content-Based Video Retrieval (CBVR) system for CCTV surveillance videos

The inherent nature of image and video and its multi-dimension data space makes its processing and interpretation a very complex task, normally requiring considerable processing power. Moreover, understanding the meaning of video content and storing it in a fast searchable and readable form, requires taking advantage of image processing methods, which when running them on a video stream per query, would not be cost effective, and in some cases is quite impossible due to time restrictions. Hence, to speed up the search process, storing video and its extracted meta-data together is desired. The storage model itself is one of the challenges in this context, as based on the current CCTV technology; it is estimated to require a petabyte size data management system. This estimate however, is expected to grow rapidly as current advances in video recording devices are leading to higher resolution sensors, and larger frame size. On the other hand, the increasing demand for object tracking on video streams has invoked the research on Content-Based Image Retrieval (CBIR) and Content-Based Video Retrieval (CBVR). In this paper, we present the design and implementation of a framework and a data model for CCTV surveillance videos on RDBMS which provides the functions of a surveillance monitoring system, with a tagging structure for event detection. On account of some recent results, we believe this is a promising direction for surveillance video search in comparison to the existing solutions

Gaussian probabilistic confidence score for biometric applications

We propose a quick and widely applicable approach for converting biometric identification match scores to probabilistic confidence scores, resulting in increased discrimination accuracy. This approach builds on a confidence scoring approach for Binomial distributions resulting from Hamming distances (commonly used in iris recognition). We derive a Gaussian confidence scoring approach that is three orders of magnitude faster than the Binomial approach while still resulting in higher recognition rates. Gaussian distributions are also more common and thus more widely applicable to different biometric systems. For probe-to-gallery (1-to-N) identification of the face recognition system tested, this approach has been shown to improve the identification rate from 25.66% to 68.05% at 1.00% false alarm rate for a CCTV video matching dataset, and from 63.34% to 73.14% for images from the LFW dataset. A sensitivity analysis demonstrates that modeling errors in genuine and impostor distributions only negatively impacts discrimination when the distribution means are modelled to be closer together than the true underlying distributions. For the reverse case where the distribution means are modeled to be further apart than the true distributions, discrimination accuracy is improved

An appearance-based approach to assistive identity inference using LBP and colour histograms

Robust identity inference is one of the biggest challenges in current visual surveillance systems. Although, face is an important biometric for generic identity inference, it is not always accessible in video-based surveillance systems due to the poor quality of the video or ineffective viewpoints where the captured face is not clearly visible. Hence, taking advantage of additional features to increase the accuracy and reliability of these systems is an increasing need. Appearance and clothing are potentially suitable for visual identification and tracking suspects. In this research we present a novel approach for recognition of upper body clothing, using local binary patterns (LBP) and colour information, as an assistive tool for identity inference